Objective: To investigate concordance between the Awareness Questionnaire (AQ) and Self-Awareness of Deficits Interview (SADI) for identifying impaired self-awareness in the community after traumatic brain injury.

Design: A retrospective file audit was conducted to retrieve data on the AQ and SADI for participants with traumatic brain injury involved in previous studies on community-based outcomes. Concordance between the AQ and SADI was examined using receiver operating characteristic curves for different scores on each measure.

Participants: A total of 80 individuals with moderate-to-severe traumatic brain injury (81% male, mean age 2.25 years, standard deviation (SD) 11.5 years) living in the community.

Methods: Participants were administered the AQ and SADI and their significant other completed the AQ and SADI checklist.

Results: Consistency of classification of impaired self-awareness between the AQ and SADI was high (i.e. 80–84%) based on scores reflecting good sensitivity and specificity. Corresponding scores on the 2 measures and rates of impaired self-awareness were as follows: SADI > 3~AQ discrepancy > 4 (45–48% with impaired self-awareness); SADI > 4~AQ discrepancy > 9 (26–34% with impaired self-awareness); SADI > 5~AQ discrepancy >12 (13–26% with impaired self-awareness).

Conclusion: The AQ and SADI yielded consistent information regarding the presence of impaired self-awareness in community-based individuals with traumatic brain injury. The choice of measure may depend on how the tool is being used to guide rehabilitation planning.

Accepted Feb 5, 2019; Epub ahead of print Feb 25, 2019

J Rehabil Med 2019; 51: 00–00

Correspondence address: Tamara Ownsworth, School of Applied Psychology, Griffith University, Mt Gravatt, 4122 Australia. E-mail: t.ownsworth@griffith.edu.au

People often do not recognize changes in their abilities or skills after traumatic brain injury. Such problems with self-awareness can lower people’s motivation to take part in rehabilitation. Two common approaches used to assess self-awareness include a brief questionnaire (e.g. 5 min) and a longer interview process (e.g. 20 min). This study aimed to identify the consistency in information and opinions formed about self-awareness from these 2 approaches. The Awareness Questionnaire and Self-Awareness of Deficits Interview were administered to 80 people with traumatic brain injury. Their family members also provided information about the abilities of the person with brain injury. Overall, there was good consistency in information gained about self-awareness between these measures. These findings suggest that similar information can be gained from a questionnaire or interview; however, the choice of measure may depend on how the tool is being used to guide rehabilitation planning.

Impaired self-awareness (ISA) or the inability to accurately perceive post-injury impairments often reduces engagement in rehabilitation after traumatic brain injury (TBI). ISA can persist well beyond hospital discharge and influences community reintegration. Geytenbeek and colleagues (1) found that 69% and 54% of individuals with TBI displayed ISA at discharge and 6 months post-discharge, respectively. ISA was associated with poorer psychosocial outcomes irrespective of injury severity.

Given the adverse long-term effects of ISA, there has been considerable focus on interventions to improve self-awareness. Metacognitive interventions involving systematic feedback (e.g. pre-task predictions, audio-visual recordings) on functional tasks are considered a practice standard for treating ISA after TBI (2), with several randomized controlled trials supporting clinical efficacy (3, 4). As these interventions are more time- and resource-intensive than routine assessment feedback, reliable approaches for identifying ISA are needed to guide decision-making regarding the need for metacognitive interventions.

The Self-Awareness of Deficits Interview (SADI) (5) is a validated semi-structured interview that uses open-ended questions and prompts to comprehensively examine individuals’ understanding of post-injury deficits (subscale 1), functional consequences of impairments (subscale 2) and realistic goal-setting (subscale 3). Subscale scores range from 0 (no impairment) to 3 (severe ISA), resulting in a total score of 0–9. The SADI assesses both the presence and nature of ISA across diverse areas of functioning. In a cluster analysis of long-term community outcomes after brain injury (6), SADI scores ≤3 distinguished between poor self-awareness (mean 3.9, standard deviation (SD) 1.7) and good self-awareness (mean 1.4, SD 1.1) groups, with the former group experiencing significantly poorer global psychosocial outcomes. Yet, administration time (20–30 min) may reduce feasibility of use in clinical practice.

The Awareness Questionnaire (AQ) (7) is a brief (5–10 min) questionnaire that assesses ISA by comparing self-ratings and significant-other ratings of changes in sensory/motor, cognitive and behavioural/affective functions. In an inpatient sample, Sherer et al. (7) used logistic regression to identify cut-off points for ISA on the AQ, based on post-discharge employment status. They proposed the following discrepancy scores and clinical interpretations: < 20 mild or no ISA, 20–29 = moderate ISA, and > 29 = severe ISA. Given that most individuals demonstrate some level of ISA early after TBI (8), large discrepancies may be warranted to reliably indicate ISA in an inpatient setting. To guide assessment of self-awareness in the community this study aimed to examine concordance between the AQ and SADI for identifying ISA in individuals with long-term TBI.

A file audit was conducted to retrieve AQ and SADI data for TBI participants involved in research on psychosocial outcomes (9) and rehabilitation (10, 11). Following ethics approval from hospital and university ethics committees, 150 participants with moderate-to-severe TBI (post-traumatic amnesia (PTA) > 24 h or Glasgow Coma Scale (GCS) score < 13) were recruited from metropolitan-based brain injury outpatient rehabilitation and community-based services (2004–17) across 3 studies. Of these, 30 were excluded due to non-traumatic aetiology (e.g. stroke), 15 sustained mild TBI, and 25 were missing significant-other data on the SADI and/or AQ. Eighty individuals with moderate-to-severe TBI, aged 18–57 years (M=32.25, SD=11.5 years), had previously completed the AQ and SADI within the same assessment session and were included in this study. They were predominantly male (81%) and the mean time since injury was 3.60 years (SD 2.9; range 0.25–10 years). The main cause of TBI was motor vehicle or other traffic accidents (65%), followed by falls (16%), sporting or work accidents (11%) and assaults (8%). The mean length of PTA was 59.13 days (SD 108.4 days) and GCS was 6.92 (SD 4.2).

The AQ (7) asks respondents to compare a person’s abilities before and after TBI on 17 items (1 = much worse to 5 = much better), producing a total score of 17–85. Positive discrepancy scores (self-ratings minus significant-other ratings) indicate under-reporting of impairments. The SADI (5) was administered by trained researchers who scored the interview independently of the AQ. A significant other (predominantly family members) completed a checklist concerning the person’s post-injury changes to assist with scoring.

Data were analysed using the Statistical Package for the Social Sciences (SPSS), version 24. Concordance between the SADI and AQ was examined using Spearman’s rho and by constructing receiver operating characteristic (ROC) curves to examine consistency in classification of ISA at different scores on each measure. For these analyses the total score on the SADI  ≥ 2 to ≥ 6) was selected as the state variable due to the smaller range of possible scores than the AQ and previous research supporting the concurrent validity in the community setting (6). Sensitivity (true-positive rate) represents the proportion of individuals classified as having ISA on the SADI and correctly identified by the AQ. Specificity (true negative rate) is the proportion classified as not having ISA on the SADI (< 3/9) and correctly identified by the AQ. Consistency of classification of ISA between the AQ and SADI was calculated, with scores > 0.80 considered “good” (10).

SADI total scores ranged from 0 to 9 (mean 2.5, SD 2.09) and AQ discrepancies ranged from 18 to –34 (mean 5.49, SD 9.96). Scores on the AQ and SADI were strongly associated (rho = 0.65, p < 0.001). Table I presents the ROC curve data for the SADI (state variable: ≥ 2 to ≥ 6) and AQ discrepancy scores ranging from ≥ 2 to ≥ 13. Scores in bold font represent the optimal balance between sensitivity and specificity. Rates of ISA were comparable at corresponding scores on the 2 measures, as follows: SADI ≥ 2~AQ discrepancy ≥ 3 (56–60% with ISA); SADI ≥ 3~AQ discrepancy ≥ 4 (45–48% with ISA); SADI ≥ 4~AQ discrepancy ≥ 9 (26–34% with ISA); SADI>5~AQ discrepancy ≥ 12 (13–26% with ISA).

Table II presents the consistency in classification of ISA between the SADI and AQ, based on scores with optimal levels of sensitivity and specificity. SADI and AQ scores of ≥ 2 yielded very high rates of ISA (59–60%) and classification consistency was only 76.3%. Sensitivity, specificity and classification consistency improved to 80% for SADI ≥ 3 and AQ ≥ 4, with 45% and 48% classified as having ISA on each tool, respectively. As shown in Fig. 1, the area under the curve (AUC) in the ROC curve analysis was 0.88 (95% confidence interval (95% CI) 0.80~0.96). Classification consistency was also good at higher scores on each measure, as follows: SADI ≥ 4~AQ discrepancy > 9 (83%); SADI > 5~AQ discrepancy ≥ 12 (84%). Slightly higher rates of ISA were identified by the AQ at each corresponding SADI score.

Good concordance between the AQ and SADI indicated that the tools yield consistent information regarding the presence of ISA in community-based individuals. A 4-point AQ discrepancy corresponded to the SADI score (≥ 3) that previously distinguished between good and poor self-awareness groups and community outcomes (6). Classification consistency was also high when adopting more conservative scores for ISA; namely, SADI ≥ 4~AQ ≥ 9 or SADI ≥ 5~AQ ≥ 12.

These AQ discrepancies are smaller than ISA cut-offs (> 20) in an inpatient sample (7). Given that self-awareness typically improves following hospital discharge (1, 8), smaller AQ discrepancies may be more sensitive to detect persisting ISA in a community sample. Accurate identification of ISA is imperative to guide appropriate management and use of clinical resources. Due to its brevity, the AQ is more feasible for routine administration to detect ISA and determine the need for metacognitive interventions. The SADI could be used as a more comprehensive assessment to determine the nature of awareness deficits (i.e. specific to a particular functional domain or generalized across multiple domains) and the extent to which individuals understand the implications of their deficits and can set realistic goals. Due to the semi-structured interview format the SADI also offers opportunity for rapport building, observation of emotional responses, and asking patients and significant others for their perspective rather than focusing on standard set of questions (12).

Such information can greatly assist with client-centred goal-setting and tailoring metacognitive interventions; for example, an intervention may target self-awareness of memory problems that affect progress on the goal to live independently. Feedback on occupational performance on goal-salient activities can enhance self-awareness and self-regulation skills (3, 4, 13).

Due to the convenience sample and potential for selection bias, caution is needed in broadly generalizing the findings to the broader TBI population. Participants in the current sample were more likely to have been admitted to a major metropolitan hospital than a rural or regional hospital, and hence due to their locality were able to access specialist brain injury outpatient and community-based rehabilitation. Therefore, the findings are likely to be most applicable to patients accessing metropolitan-based outpatient or community rehabilitation. It is also important to acknowledge the reliance on significant others/family members reports for comparison with patients’ self-report, as is typical for measurement of self-awareness. The potential biases associated with significant others’ reports have been well documented in the literature (8). As there is no “gold standard” for measuring ISA, the SADI was used as the state variable in the ROC analysis due to research supporting its concurrent validity in the community setting (6). It is important to note that, although classification consistency was good using a 4-point AQ discrepancy, 20% were incorrectly classified as having or not having ISA. Further research is needed to determine the predictive validity of AQ and SADI cut-off scores for community re-integration outcomes.

The AQ and SADI yielded consistent information regarding the presence of ISA in community-based individuals with TBI. The AQ may be more feasible for routine administration to detect ISA, whereas the SADI provides a more comprehensive assessment of the nature of ISA and capacity to set realistic goals.

Data collection in this study was funded by 2 grants from the National Health and Medical Research Council (APP1083064 & APP210347).